1. Field of the Invention
The present invention generally relates to testing of programs and software such as kernels and drivers and, more particularly, to frameworks for injecting faults into or otherwise simulating errors in test code paths of software to assess the appropriateness of exception handling by the software.
2. Relevant Background
Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. Although crucial to software quality and widely deployed by programmers and testers, software testing still remains an art, such as due to sometimes limited understanding of the principles of software. The purpose of testing can be quality assurance, verification and validation, reliability estimation, and/or the like. Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major areas of testing. Software testing is often a trade-off between budget, time and quality.
For instance, one part of software testing may include instrumentation (e.g., monitoring, tracing, etc.) of exception handling by the software and/or operating system (i.e., the process of responding to the occurrence, during computation, of exceptions or, in other words, anomalous or exceptional events that require special processing) to gauge how the software responds to the exceptions. To improve the coverage of a test, test engineers often manually introduce faults into test code paths that might otherwise rarely be followed. For instance, compile-time injection involves modifying the source code to inject simulated faults into a system while runtime injection involves use of a software trigger to inject a fault into a running software system.
Tracing typically involves recording information about the execution of the software (e.g., after fault injection) such as recording a timestamp that indicates when a particular function was called, the process running when the function was called, etc. In one arrangement, a kernel tracing framework may provide a set of programs and application program interfaces (APIs) that may be used to present an overview of the performance of any part of the software as well as to debug or observe operations within the software. When the tracing framework is activated, a tracing event is recorded in any appropriate manner and/or location (e.g., buffer) whenever an execution thread executing in the system encounters a trace point (i.e., a point in the execution path of the thread at which data about the state of the software is collected).
For instance, DTrace® is a comprehensive dynamic tracing framework created by Sun Microsystems that can be used to observe the runtime behavior of user level programs (and of the operating system itself) for troubleshooting kernel and application problems on production systems in real time. Broadly, DTrace can be used to obtain a global overview of a running system (e.g., such as the amount of memory, CPU time, filesystem and network resources used by the active processes) as well as more fine-grained information (e.g., such as a log of the arguments with which a specific function is being called, a list of the processes accessing a specific file, etc.).
FIG. 1 illustrates a schematic block diagram of a DTrace architecture or framework 10 according to the prior art. As shown, the framework 10 broadly includes a plurality of DTrace providers 14 (e.g., kernel modules such as syscall, profile, sdt, etc.) in kernel space (e.g., of memory of a computer or computing system, see FIG. 6) that publish a plurality of DTrace “probes” 18. A probe 18 is a location or activity to which the framework 10 can bind a request to perform actions, such as logging the system calls or function calls in user level processes, recording a stack trace, recording timestamps, and so on. A probe is said to fire when the activity, as specified by the probe description, takes place. In this regard, each probe 18 may be likened to a sensor that may be inserted into the code of a program or software at an instrumentation point (e.g., in or adjacent a test code path) to monitor, record and report back on exception handling. Unless the probes 18 are used, the probes 18 have a zero impact on the performance of the application or the system as a whole.
Once the probe points are integrated, the application code is compiled and the resulting binary file is run, the probes 18 become available for consumption by DTrace user level consumers 22. A DTrace consumer 22 is any process that interacts with the framework 10 and specifies the instrumentation points (the points in the code where the probes are inserted) by specifying probe descriptions. The framework 10 also includes a library interface 26 used by the consumers 22 to access the probes 18 through a DTrace kernel driver 30.